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(54) Video management systems 

(57) A videostream broadcast or recording includes 
a relevance level indicator. When a user is unable to 
view a portion of a videostream, the user may record a 
highlight videostream. The highlight video is assembled 
by selecting portions of the videostream for recordation 



based on the relevance Indicatortransmltted in conjunc- 
tion with the basic videostream. The user may hen re- 
play highlights of the basic videostream at a later time, 
without the need for manual editing of the videostream, 
or for reviewing the entire video stream. 
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Description 

The present invention relates to video management 
systems, and in particular, but not exclusively to a sys- 
tem and method for selective replay of a videostream. 

Videostreams are often viewed by a video system 
user in real time. That is, the user views the videostream 
as it is broadcast, or even as the event actually occurs. 
Typical videostreams include those broadcast over 
mass media television. These videostreams include 
sporting events (e.g. a baseball game), political debates 
(e.g. a Senate or committee hearing), news coverage 
(natural disasters, and broadcasts about other events 
of interest). 

Other video applications are beginning to emerge. 
As computers become more prolific and as data trans- 
mission systems become more sophisticated, it is ex- 
pected that the line between conventional television and 
computers will become more blurred. For example, sys- 
tems already exist for playing video on a computer 
screen, and transmitting videostreams over a computer 
network for video conferencing. Conversely, television 
systems already exist that can display video on demand, 
provide sophisticated schedule data sorting, and other 
traditionally computer-related functions. 

In any of these video applications, a user may not 
desire to view the videostream continuously. As a simple 
example, viewers of a video event such as a sporting 
event are often drawn away from the event to answer, 
for example, telephone calls or to cook dinner. Often a 
user wishes to be brought "up-to-date" on the events 
that take place in the user's absence without the need 
to review, for example, a video tape covering the entire 
user's absence. 

Video replay and editing systems are well known to 
those of skill in the art. It is well known that manual ed- 
iting can be performed sometime after the event, and a 
manually edited tape replayed at a later time. Such sys- 
tems have in fact become quite sophisticated. Typical 
examples are disclosed in U.S. Patent Nos. 5,109,482 
and 5,029,013. Such systems suffer from a variety of 
limitations, however. These limitations include the time 
delay required for the manual editing to take place. Fur- 
ther, such editing systems are often directed to masses 
of users, and may not be responsive to the desires of a 
particular user. 

From the above it is seen that an improved videos- 
tream management system is desirable. 

An improved videostream management system can 
be is provided by embodiments of the present invention. 
According to a specific embodiment of the invention, a 
videostream broadcast or recording includes a rele- 
vance level indicator. Such level indicators often will 
take the form of an "excitement" level in the most imme- 
diate applications. When a user is unable to view a por- 
tion of a videostream the system may record a highlight 
video. The highlight video may be assembled by select- 
ing portions of the videostream for recordation based on 



the relevance indicator transmitted in conjunction with 
the basic videostream. The user may then replay high- 
lights of the basic videostream at a later time. Thus, one 
can create a highlight video without the need for manual 
5 editing of the videostream, or for reviewing the entire 
video stream. 

Another embodiment the invention provides a 
method of recording and replaying a videostream in- 
cluding the steps of receiving a basic videostream; in 
association with the basic videostream, receiving a rel- 
evance indicator; forming a highlight videostream re- 
cording, the highlight videostream recording including 
selected portions of the basic videostream, the selected 
portions selected based on the relevance indicator; and 
playing the highlighted videostream for a user. 

For a better understanding of the invention, and to 
show how the same may be carried into effect, reference 
will now be made, by way of example, to the accompa- 
nying drawings, in which:- 

Fig. 1 is an overall block diagram of a video system 
according to one embodiment of the invention; 
Fig. 2 illustrates selection of a portion of a videos- 
tream for recordation and playback; and 
Fig. 3 illustrates the playback process in greater de- 
tail. 

Embodiments of the present invention can provide 
improved videostream management systems, and are 
applicable to computer systems, television systems, or 
combinations of such systems. 

Suppose a user is watching a particular videos- 
tream such as a sporting event. In embodiments of the 
present invention, the user indicates to the system that 
he/she wishes to stop watching an ongoing event, but 
wishes to return to the event later to review highlights of 
the videostream which will be missed during the user's 
absence. The system records selected portions of the 
videostream during the user's absence based on a rel- 
evance level transmitted with the videostream. 

When the user returns, a portion of the videostream 
has been recorded, and may be reviewed at the user's 
leisure. The system desirably presents the user with a 
menu showing, among other things, the number of 
events that exceed a set relevance level, the number of 
minutes of video exceeding a set relevance level, and 
how many minutes the user desires to use to review the 
missed video. In the simplest embodiments, the system 
may simply play back all video above a threshold rele- 
vance level. In other embodiments the system may ad- 
just the relevance level of the video to be played to "fit" 
it into a fixed playback time. 

Suppose, for example, that the user has missed x 
minutes of video. Suppose the user will use y minutes 
to review the missed video while the next portion of the 
videostream is received. The system will, in a preferred 
embodiment, select highlights of x+y minutes of video. 
That is, the system will continue recording while the 
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highlights are displayed. This will allow the user to 
"catch up" on the missed portion of the video as well as 
highlights of the extra time used for review. 

In one embodiment, the system will assume that the 
y minutes of new video will, on the average, have the 
same relevance level as the x minutes of missed video. 
Therefore, the highlight video would initially be assumed 
to include x/(x+y) percent of the x minutes of missed vid- 
eo and y/(x+y) percent of the video transmitted during 
review of the highlights. In a preferred embodiment, the 
system will continuously adjust this ratio such that if, for 
example, little of relevance is happening during the y 
minutes of review, the system may display more of the 
X minutes of missed video. On the other hand, if the sys- 
tem becomes aware that a greater than expected 
amount of relevant material is being transmitted, the 
system may transmit less of the x minutes of video. The 
threshold relevance level for video to be displayed may 
also be adjusted to fit the highlights into the allotted time. 

The relevance level is preferably transmitted with 
the videostream, but may be broadcast over a separate 
medium. For example, in one embodiment the videos- 
tream is a conventional NTSC video stream, and the rel- 
evance level is broadcast in the vertical blanking inter- 
val. Such technology is available using, for example, the 
Intel "Intercast" standard. In other embodiments, the 
videostream is broadcast over a digital medium or over 
a computer network. In such embodiments the rele- 
vance level may be broadcast in control or overhead bits 
in the digital videostream, over a cellular network, pag- 
ing network, FM broadcast, or the like. Also, the rele- 
vance level may be generated within a receiver auto- 
matically based on an evaluation of the programming 
content. Alternatively the PAL or SECAM video systems 
can be utilised. 

The relevance level may take any of a number of 
forms. In one example, the relevance level is added by 
the broadcaster. The added relevance level is added by 
the broadcast network manually. In some cases, such 
as in the case of a sporting event, the relevance level 
may be added automatically either at the transmitter or 
receiver end. In the case of a sporting event, the crowd 
noise level will often be one indication of the excitement 
level of the sporting event and will be used as a rele- 
vance level (adding, preferably a time before and/or af- 
ter a set noise level for display in addition to the time for 
which a noise level is exceeded). Accordingly, the rele- 
vance level will automatically increase as the noise level 
of the crowd at the sporting event increases. A buffer 
period may be added both before and after any such 
increases in noise level to allow context to be added to 
a particular event of interest. 

In still other cases, the relevance level may take the 
form of a rating guide with, for example, material unsuit- 
able for viewing by children carrying a low relevance lev- 
el while suitable material carries a high relevance level. 
In this way, a recording system can automatically excise 
scenes including mature content. 



The service provided herein may be provided as an 
optional sen/ice to a subscriber of a particular video 
service. In the case of conventional television signals, 
the user may subscribe to the highlight service through 

5 a cable service provider. Subscription-based services 
for optional services are known in the art and include 
television guide services such as disclosed in Young et 
al., U.S. Patent No. 5,353,121. See also, U.S. Patent 
No. 5,335,277, incorporated herein by reference for all 

10 purposes. 

Fig. 1 is a block diagram of a videostream manage- 
ment system 100 according to one embodiment of the 
present invention. A videostream source 101 transmits 
a videostream 103 and a relevance level 105. The rel- 

15 evance level may be a measure of "excitement" in the 
videostream, the level of "interest" in, for example, an 
educational program, or any other measure of interest 
to a particular user or group of users. Multiple measures 
of relevance may be transmitted, in which case the user 

20 may select the desired measure of relevance, or level 
of relevance. In some cases, "default high" or "default 
low" measures may be applied by the broadcaster, such 
as in advertising such that a user is forced to view at 
least a portion of the advertising in a particular video 

25 broadcast. 

The transmission of the broadcast stream may be 
from a remote site, or from the user site. For example, 
the broadcast station may be a cable head end, a sat- 
ellite broadcast site, a hotel videotape machine, or a vid- 

30 eo source on a computer network. The user site may be 
an individual viewer or a group of viewers. 

The user site receives the videostream and the rel- 
evance level at a system 1 07. A selection system 1 09 
(using, for example, a microprocessor programmable 

35 logic device) determines if the user has set the system 
into a mode for recordation of highlights. If not, the vid- 
eostream is passed directly to the display 1 1 1 which may 
be adapted, for example, to display an NTSC videos- 
tream. If the user has set the system into a mode for 

40 recordation of highlights, the videostream or parts of the 
videostream are passed to a recording device 113. Re- 
cording device 113 is, in some embodiments, a video- 
tape recorder or optical disk system. In other embodi- 
ments, particularly those related to the distribution of a 

45 videostream in a computer network, the recording de- 
vice is a computer hard disk or optical disk. Software for 
performing the operations herein is stored on memory 
1 1 0 such as a ROM, RAM, EPROM, or magnetic or op- 
tical disk. 

50 The selection system may select all of the video for 
recordation, and select among the recorded elements 
for playback at a later time, or it may select only ele- 
ments having an interest level above a determined level 
for recordation. In the latter case, further filtering of the 

55 recorded material may take place when the recorded 
material is played back. 

Fig. 2 illustrates the process of recording and play- 
ing back selected elements of a videostream according 
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to relevance level in one embodinnent of the invention. 
At step 201 the systenn deternnines if the user desires 
the record/playback features. For example, the user 
may plan to be absent. If the features are desired, the 
currently received portions of the videostreann become 
eligible for recording if their relevance level is high 
enough- If not, the system waits until there is an indica- 
tion that the features are desired. The system may ob- 
tain input that the record/play back features would be de- 
sired from a variety of sources such as from a computer 
mouse, keyboard, a remote control device, or by visually 
observing the user depart the viewing area through an 
attached video camera. 

At step 203 the system reads the relevance level of 
the videostream at a given time, and if the relevance 
level exceeds a preset value, records the videostream 
at step 205. In most embodiments the recording will take 
place for a fixed period of time (e.g. 1 minute) and the 
system will again determine if recording is still desired 
at step 201 . When recording is no longer desired, for 
example, the user is no longer absent, and a check at 
step 207 determines that material has been recorded 
the system begins a playback process at step 209. After 
the playback is complete, the system again begins the 
process as previously discussed at step 201 . 

Fig. 3 illustrates the playback process at step 207 
in greater detail. At step 301 the system inputs the length 
of the recording process of step 205, e.g. the number of 
minutes of video that have been missed. At step 303 the 
system determines the number of minutes available for 
playback based on user input or a default value. At step 
305 the system calculates the number of minutes avail- 
able to playback the missed video according to the for- 
mula y*(x/(x+y)). At step 307 the system adjusts the rel- 
evance level of the material to be played back down- 
wards if necessary so that the already recorded material 
will fit into the calculated number of minutes available. 

At step 309 the system begins playback, playing 
back only the material exceeding the relevance level set 
at step 307 such that the video will fit in the allotted per- 
centage of the y minutes available. If the relevance level 
of the y minutes of video is the same as the x minutes 
of video, the highlights should fit directly into the y min- 
utes available. However, at step 311 the system moni- 
tors the new material and, at step 31 3 the relevance lev- 
el of material to be played back is adjusted if necessary. 
For example, if the relevance level goes up during play- 
back, the system will gradually bring the necessary level 
for playback of material up such that less of the early 
material is played. In some embodiments a high thresh- 
old level is set above which the system will switch to 
"real time" if the current incoming video exceeds a se- 
lected value. In this way, events of very high interest are 
seen immediately despite the playback process. 

The above description is illustrative and not restric- 
tive. Many variations of the invention will become appar- 
ent to those of skill in the art upon review of this disclo- 
sure. Merely by way of example the invention is illustrat- 



ed primarily with respect to television signals, but the 
invention is not so limited and may be applied to com- 
puter systems and others. 

5 

Claims 

1 . A method of recording and replaying a videostream 
comprising the steps of: 

10 

receiving a basic videostream; 
in association with said basic videostream, de- 
veloping a relevance level; 
forming a highlight videostream recording, said 
15 highlight videostream recording comprising se- 

lected portions of said basic videostream, said 
selected portions selected based on said rele- 
vance level; and 

playing said highlight videostream for a user. 

20 

2. The method as recited in claim 1 wherein said step 
of receiving a videostream is a step of receiving an 
NTSC videostream, 

25 3. A method as claimed in claim 1 or 2, wherein said 
relevance level broadcast is in a vertical blanking 
interval of said videostream. 

4. The method as recited in claim 1 , 2 or 3, wherein 
30 said relevance level is a measure of excitement of 

content of said videostream. 

5. The method as recited in claim 1 , 2, 3 or 4, wherein 
said selected portions are selected by the steps of: 

35 

identifying a time period of eligibility for record- 
ing; 

identifying a time period for review; and 

based on said time period of eligibility for re- 
40 cording and time period for review, selecting 

video for playback. 

6. The method as recited in claim 5, wherein identify- 
ing a time period of eligibility for recording compris- 

45 es identifying a time period of missed video. 

7. The method as recited in claim 5, wherein identify- 
ing a time period of eligibility for recording compris- 
es identifying a time period that a viewer was ab- 

50 sent. 

8. The method as recited in claim 5, wherein selecting 
comprises evaluating a threshold relevance level 
for playback according to x/(x+y) where x is said 

55 time of eligibility for recording and y is said time for 
review. 

9. The method as recited in claim 6, wherein selecting 
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further comprises adjusting said tinreshold rele- 
vance level for playback in accordance with varia- 
tions in the relevance level of the videostream re- 
ceived during said time period for review. 

10. The method as recited in any one of claims 1 to 9, 
wherein developing comprises receiving said rele- 
vance level as generated by a video broadcaster. 

11. The method as recited in any one of claims 1 to 9, 
wherein developing comprising automatically eval- 
uating said relevance level based on content of said 
videostream. 

12. The method as recited in any one of the preceding 
claims, wherein a threshold relevance level for play- 
back is determined based on a time available for 
playback. 

13. The method as recited in claim 12, wherein said 
threshold relevance level for playback is adjusted 

during playback. 

14. The method as recited in any one of the preceding 
claims, wherein portions of said videostream above 
a selected relevance level within a first time period 
and a second time period are played back during 
said second time period. 

1 5. The method as recited in claim 1 4, wherein said first 
time period is a missed time period. 

16. The method as recited in claim 14 or 15, wherein 
said portions of said videostream above an elevat- 
ed reference level above said selected reference 
level are displayed in real time during said second 
time period. 

17. The method as recited in any one of the preceding 
claims, wherein said highlight videostream is re- 
corded while said user is absent from a video dis- 
play device. 

18. A method of transmitting videostream comprising 
the steps of: 

transmitting a basic videostream; and 
transmitting a relevance level in association 
with said basic videostream. 

19. A method as claimed in claim 18, further compris- 
ing: 

forming a highlight videostream recording, said 
highlight videostream recording comprising se- 
lected portions of said basic videostream, said 
selected portions selected based on said rele- 
vance level; and 



playing said highlight videostream for a user. 

20. A highlight video system comprising: 

5 a video display; 

a receiver for receiving a videostream; 
a receiver for receiving a videostream rele- 
vance level; and 

a recorder, said recorder recording or playing 
10 back selected highlight video from said videos- 

tream on said video display, said selected high- 
light video having a selected relevance level. 

21. A video system comprising: 

15 

a receiver for receiving a videostream; 
a receiver for receiving a videostream rele- 
vance level; and 

means for controlling a recorder in dependence 
20 upon the received relevance level. 

22. A video system as claimed in claim 21 , further com- 
prising a recorder for recording and playing back se- 
lected highlight video from such a received videos- 

25 tream, the highlight video having a selected rele- 
vance level. 

23. A video system according to claim 20, 21 or 22, 
wherein said video display is adapted to display an 

30 NTSC videostream. 

24. A video system according to any one of claims 20 
to 23, wherein said receiver for receiving a videos- 
tream relevance level is an NTSC vertical blanking 

35 interval receiver. 

25. A video system according to any one of claims 20 
to 24, further comprising a processor for selecting 
a relevance level for playback during playback of 

40 highlight video. 

26. The video system according to any one of claims 
20 to 25, wherein said recorder is a videotape re- 
corder. 

45 

27. The video system according to any one of claims 
20 to 25, wherein said recorder is an optical or mag- 
netic disk. 

50 28. A storage medium conveying software for perform- 
ing the steps of: 

inputting a relevance level of an incoming vid- 
eostream; and 

55 selecting portions of said videostream where 

said portions exceed a selected relevance lev- 
el. 
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29. A storage medium as claimed in claim 28, wherein 
said software for selecting portions of said videos- 
tream is configured to: 

identify a time period of receiving video eligible s 
for review; 

identify a time period for review; and based on 
said time period of receiving video eligible for 
review and said time period for review, select- 
ing video for playback io 

30. A storage medium as claimed in claim 29, wherein 
said time period of receiving video eligible for review 
is a time period of missed video. 

15 

31 . A storage medium as claimed in claim 28, 29 or 30, 
wherein said relevance level is an excitement level. 

32. A storage medium as claimed in claim 28, 29, 30 or 

31 , wherein said software for selecting determines 20 

which video to select based on a relationship of the 
form x/(x+y) where x is a time of missed video and 
y is a time of playback. 

33. A storage medium as claimed in claim 28, 29 or 30, 25 
wherein said selected relevance level is selected by 

said software. 

34. A method of selecting highlight video for playback 
comprising the steps of: 30 

inputting a crowd noise level of an event; and 
playing back selected, recorded highlights of 
said event, said highlights selected based on 
said crowd noise level. 35 

35. The method as recited in claim 34, wherein said se- 
lected highlights are those where said crowd noise 
level exceeds a value. 

40 

36. The method as recited in claim 35, wherein said val- 
ue is preset. 

37. The method as recited in claim 35, wherein said val- 
ue is user defined. 45 

38. The method as recited in claim 35, 36 or 37, wherein 
said event is a sporting event. 
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